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Abstract 

We analyze the performance of RiskMetrics, a widely used methodology for mea- 
suring market risk. Based on the assumption of normally distributed returns, the 
RiskMetrics model completely ignores the presence of fat tails in the distribution 
function, which is an important feature of financial data. Nevertheless, it was com- 
monly found that RiskMetrics performs satisfactorily well, and therefore the tech- 
nique has become widely used in the financial industry. We find, however, that the 
success of RiskMetrics is the artifact of the choice of the risk measure. First, the 
outstanding performance of volatility estimates is basically due to the choice of a 
very short (one-period ahead) forecasting horizon. Second, the satisfactory perfor- 
mance in obtaining Value-at-Risk by simply multiplying volatility with a constant 
factor is mainly due to the choice of the particular significance level. 
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1 Introduction 



Risk management is one of the top priorities in the financial industry today. A 
huge effort is being invested into developing reliable risk measurement meth- 
ods and sound risk management techniques by academics and practitioners 
alike. J. P. Morgan was the first to develop a comprehensive market risk man- 
agement methodology based on the Value-at-Risk (VaR) concept [1]. Their 
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product, RiskMetrics [2] has become extremely popular and widespread. It 
has greatly contributed to the dissemination of a number of basic statisti- 
cal risk measurement methods and to the general acceptance of VaR as an 
industry standard. Although backtests performed first by J. P. Morgan and 
later by other market participants lent support to the RiskMetrics model, its 
basic assumptions were shown to be questionable from several points of view 
(see e.g. [3]). Moreover, the existence of fat tails in real market data (see [4,5] 
for a discussion) is in a clear conflict with RiskMetrics' assumption of nor- 
mally distributed returns, which can lead to a gross underestimation of risk. 
Furthermore, serious doubt has recently been raised as to the stability and 
information content of the empirical covariance matrices used by the model 
for calculating the risk of portfolios [6-8]. 

Accurate evaluation of risks in financial markets is crucial for the proper as- 
sessment and efficient mitigation of risk. Therefore, it is important to see to 
what extent widespread methodologies like RiskMetrics are reliable and what 
their possible limitations are. In particular, we try to understand why, despite 
the evident oversimplifications embedded in the model, it can perform satisfac- 
torily. We will argue that the apparent success of RiskMetrics is due basically 
to the way risk is quantified in this framework, which does not necessarily 
mean that this particular risk measure is the most adequate one. 

The paper is organized as follows. In Section 2 a sketchy overview of the 
RiskMetrics methodology will be presented. In Section 3 the reasons for the 
success of RiskMetrics will be discussed. Finally, Section 4 is a short summary. 



2 Overview of the RiskMetrics Methodology 

It is well-known that daily returns are uncorrelated whereas the squared re- 
turns are strongly autocorrelated. As a consequence, periods of persistent high 
volatility are followed by periods of persistent low volatility, a phenomenon 
known as "volatility clustering" . These features are incorporated in RiskMet- 
rics by choosing a particular autoregressive moving average process to model 
the price process (see below). Furthermore, RiskMetrics makes the very strong 
assumption that returns are conditionally!^ normally distributed. Since the 
standard deviation of returns is usually much higher than the mean, the lat- 
ter is neglected in the modelj^, and, as a consequence, the standard deviation 

^ Conditional here means conditional on the information set at time t, which usually 
consists of the past return series available at time t. 

^ The typical yearly mean return in equity markets is 5%, while the typical stan- 
dard deviation is 15%. For a time horizon of one day, however, the mean becomes 
5%/250=0.02%, while the standard deviation becomes 15%/V250 Ri 0.95%, i.e. 
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remains the only parameter of the conditional probability distribution func- 
tion. In order to avoid the usual problems related to the uniformly weighted 
moving averages, Riskmetrics uses the so called exponentially weighted mov- 
ing average (EWMA) method [2, pp. 78-84] which is meant to represent the 
finite memory of the market. Accordingly, the estimator for the volatility is 
chosen to be 

-int = = (1 - A) E A^-*-., (1) 

^T=Q ^ T=n 



where A is a parameter of the model (0 < A < 1). The notation at+i\t em- 
phasizes that the volatility estimated on a given day {t) is actually used as a 
predictor for the volatility of the next day (t + 1). The daily VaR at confidence 
level p (e.g. 95%) can then be calculated (under the normality assumption) by 
multiplying (Tt+i\t with the 1—p quantile of the standard normal distribution. 
Moreover, this technique can be used to measure the risk of individual assets 
and portfolios of assets as well. For linear portfolios (i.e. containing no options) 
the usual method to obtain the volatility is to estimate the covariance matrix 
of asset returns, element-by-element, using the EWMA technique and then 
calculate the portfolio volatility as cr^ = Wi Wj aij, where Wi is the weight 
of asset i in the portfolio. Equivalently, however, one can calculate the return 
on the portfolio first, and then apply the EWMA technique directly to the 
whole portfolio [9]. Finally, the value of the parameter A is determined by an 
optimization procedure. On a widely diversified international portfolio, Risk- 
Metrics found that the value A = 0.94 produces the best backtesting results 
[2, pp. 97-101]. 



3 Why Does RiskMetrics Work? 



In order to explain the reasonably successful performance of RiskMetrics, first 
we recall the work by Nelson [10] who showed that even misspecified models 
can estimate volatility rather accurately. More explicitly, in [10] it is shown 
that if the return generating process is well approximated by a diffusion, a 
broad class of even misspecified ARCH models^ can provide consistent esti- 
mates of the conditional volatility. Since RiskMetrics can be considered as an 
IGARCII(1,1) model[3 the results of [10] offer a natural explanation for the 
success of RiskMetrics in estimating volatility. Actually, in the RiskMetrics 

about 50 times larger than the mean. 

^ See [11] for a survey on ARCH type models (e.g. ARCH, GARCH, IGARCH, 
etc.). 

^ In the IGARCH(1,1) model, returns are generated by the following stochastic 
process: rt = (Tt^ti where £t ~ i.i.d.(0,l), (t| = u) + f3i (Ji_i + ai £t_i and ai+Pi = 1. 



3 



0.5 



0.4 



0.3 



0.2 



0.1 





leptokurtic 


- 


^'^^^^^^^^^^^^^^^^ 







-1 1 




Fig. 1. Probability distribution function (PDF) of the leptokurtic Student t distri- 
bution with 7 degrees of freedom (scaled to unit variance) and the standard normal 
distribution along with vertical lines at -1.65 and -2.33 corresponding to the 5 and 
1% quantiles of the standard normal distribution. 

framework, this estimate is used as a one-day ahead volatihty forecast, never- 
theless it seems that this does not significantly worsen its accuracy. However, 
if one uses this estimate to calculate (as often required by regulators) a multi- 
period forecast using the simple "square-root-of-time" ruleQ the quality of the 
forecast is bound to decline with the number of periods. A comparative study 
of the rate of deterioration of these forecasts with time within the RiskMetrics 
model resp. other, more sophisticated volatility models is an interesting topic 
for further research. 



Let us turn to the apparent success of RiskMetrics in estimating the VaR 
now. The model sets the confidence level at 95%. The prescription to obtain 
this 5% quantile is to simply multiply the volatility estimate by 1.65 (as if 
returns were conditionally normally distributed). Why such a recipe can pass 
the backtests can be understood if one analyzes numerically the 5% quantiles 
of a few leptokurtic (fat tailed) distributions. It is very often found that despite 
the presence of fat tails, for many distributions the 5% quantile is roughly -1.65 
times the standard deviation. For example, the 5% quantile of the Student t 
distribution with 7 degrees of freedom (which is leptokurtic and has a kurtosis 
of 5 similar to the typical kurtosis of returns in financial markets) is -1.60, very 
close to -1.65, or, conversely, the -1.65 percentile is 4.6%. This is illustrated 
in Fig. 1, where the PDF's of the Student and the normal distributions are 
shown. It can be seen from the figure that the areas under the two PDF's to 
the left of the -1.65 line are roughly equal. We also analyzed the empirical 
frequencies of RiskMetrics 95% VaR violations which correspond to returns 
rt+i < —1.65 at+i\t, where crt+i\t is the RiskMetrics volatility estimate obtained 

Since from Eq. (1) (T^^l^^ = (1 - A) ^rf + A ^x=X"^ ' it follows that (t^^^^ = ^(^t\t_i + 

(1 - A)r| just as in the IGARCH(1,1) model (w = 0, /3i = A, ai = 1 - A). 

^ The rule says that cr^_^_m = yfh ■ CT^_^_^^, which is derived on the basis of the 

assumption of uncorrelated returns [2, pp. 84-88]. 
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on day t. For the 30 stocks of the Dow Jones Industrial Average (DJIA), which 
are among the most hquid stocks traded on the New York Stock Exchange, 
it was founclQ that the VaR violations frequency was 4.7 ± 0.6%, i.e. close to 
5%. This explains why RiskMetrics is usually found so successful in evaluating 
risk (which it defines as the VaR at 95% confidence). 

It is evident, however, that for higher significance levels (e.g. 99%) the effect 
of fat tails becomes much stronger, and therefore the VaR will be seriously 
underestimated if one assumes normality. For example, the 1% quantile of the 
Student t distribution considered above is -2.54, significantly larger than un- 
der the normality assumption (-2.33), while the percentile corresponding to 
-2.33 is 1.43%. This can also be seen from Fig. 1. Furthermore, for the DJIA 
data considered above, the rejection frequencies were 1.4 ±0.3%, significantly 
larger than 1%. These effects are even more pronounced for a truncated Levy 
distribution (TLD). For example, a Koponen-like [12] TLD with Levy expo- 
nent u = 1.50, scale parameter c = 0.50 and truncation parameter A = 0.17, 
which provides an excellent fit to the Budapest Stock Exchange Index (BUX) 
data, has a 5% quantile equal to 1.76a whereas the 1% quantile is already 
3.75ct! [13] 

Therefore, it can be concluded that the satisfactory performance of RiskMet- 
rics in estimating VaR is mainly the artifact of the choice of the significance 
level of 95%. However, existing capital adequacy regulations require 99% con- 
fidence, and at this level RiskMetrics systematically underestimates risk. 



4 Summary 

In this paper we analyzed the performance of RiskMetrics, perhaps the most 
widely used methodology for measuring market risk. The Riskmetrics model 
is based on the unrealistic assumption of normally distributed returns, and 
completely ignores the presence of fat tails in the probability distribution, a 
most important feature of financial data. For this reason, one would expect 
the model to seriously underestimate risk. However, it was commonly found 
by market participants that RiskMetrics performed satisfactorily well and this 
helped the method to quickly become a standard in risk measurement. Never- 
theless, we found that the success of RiskMetrics is actually the artifact of the 
choice of the risk measure: the effect of fat tails is minor when one calculates 
Value-at-Risk at 95%, however, for higher significance levels fat tails in the 
distribution of returns will make the simple RiskMetrics rule of calculating 
VaR to underestimate risk. 



° The sample consisted of the daily closing prices of the 30 stocks of the DJIA 
from August 1996 to August 2000, about 1000 data points for each stock. 
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RiskMetrics has played and continues to play an extremely useful role in dis- 
seminating risk management ideas and techniques, even if oversimplified. It 
is available free of charge and, coming with a careful documentation, it is 
completely transparent and amenable for a study hke the present one: its 
limitations can be explored and, given sufficient resources, overcome. This is 
far from being the case with the overwhelming majority of the commercially 
available risk management systems which incorporate at least as strong sim- 
plifications as RiskMetrics, but coming in the the form of "black boxes," are 
completely impossible to modify. The continuing dominance of the Gaussian 
paradigm in risk management software packages represents an industry-wide 
model risk. 
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